Interpolating Depth Panoramas Captured With Kinect for the Oculus Rift

As high quality, low-cost image sensors have proliferated in the consumer market, significant new frontiers in computer vision have become accessible. Similar advances in display technology have also made low-cost, practical virtual reality tools available to consumers. However, these developments have not yet been paired together in meaningful ways. We present our novel, low-cost virtual reality system, Oculorama, which is capable of capturing real environments and reproducing them in a virtual reality scene. Oculorama uses a Microsoft Kinect version 2 (V2) [5] to capture rich images containing both color (RGB) data and depth (D) data which are then aligned to produce a single RGB-D image. Our system combines these individual images into panoramic images, then constructs, renders, and displays 3D models in real time on an Oculus Rift Developer Kit 2 (DK2) [9]. This allows a user to move through the virtual space and see an accurate and convincing representation of the originallycaptured world. The main contributions include a real time algorithm for the continuous interpolation of perspective representations from discrete depth and color panoramas and display and interface via Oculus DK2. In addition, we design and implement a robotically automated data collection process using the Kinect V2, entirely built from inexpensive, consumergrade hardware. The end result is an end-to-end virtualization system that can be reproduced at a very low cost.